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Introduction and Overview 



This report summarizes key findings and recommendations from a Community College 
Research Center (CCRC) study designed to help community colleges develop strategies for 
improving the rate at which academically underprepared students take and pass initial 
college-level (or “gatekeeper”) courses in math and English. CCRC conducted the study at 
the request of the Virginia Community College System (VCCS) to inform the system’s 
strategic objective of improving retention and academic success for their students, 
particularly the large number of students who arrive unprepared for college-level work. The 
study examined student characteristics, course-taking patterns, and other factors associated 
with higher probabilities that students who require remediation will take and pass math and 
English gatekeeper courses. 

A detailed report on the study (Roksa, Jenkins, Jaggars, Zeidenberg, & Cho, 2009) is 
available on the CCRC website. Although the study was based on an analysis of students in 
Virginia’s community colleges, the patterns of and reasons for student success in gatekeeper 
courses may well be common to many community colleges. Moreover, the analyses CCRC 
conducted as part of the study might serve as useful models for other colleges and state 
systems interested in analyzing patterns of progression and success. This paper is intended to 
stimulate discussion and inquiry among community college educators seeking to improve 
degree completion rates for the many students who come to them poorly prepared to succeed 
in college-level work. 



Research Methods 

The dataset used by CCRC was provided by VCCS and contained information on a cohort of 
24,140 first-time college students who enrolled in a VCCS college in the summer or fall 2004 
terms. It included information on student demographics, institutions attended, placement test 
scores, and placement recommendations; transcript data on courses and grades; and 
information on educational attainment (including transferring to four-year institutions and 
earning certificates and associate degrees). Students were followed for four years, through 
the 2008 summer term. CCRC examined a range of educational outcomes for this cohort, 
including: whether students took and passed developmental courses and gatekeeper English 
and math, the number of terms students were enrolled, the number of credits they 
accumulated, and whether they earned educational awards (certificates and associate degrees) 
or transferred to a four-year institution. 

CCRC’s analyses focused in particular on educational progression of students who were in 
need of developmental education. We used three different indicators of the need for 
developmental education: students’ placement test scores; course placement 
recommendations; and whether or not students took developmental courses in reading, 
writing, or math. While the three markers were correlated, their correspondence was far from 
perfect; each provided different insights into students’ educational pathways and outcomes. 
The full report (Roksa et al., 2009) presents results separately by each of these indicators, and 
it discusses in detail the extent and implications of missing data issues that were observed for 
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test scores and placement recommendations. In this summary, we highlight the key findings 
from the more detailed analyses. 



Findings 

Finding: Half of the cohort enrolled in at least one developmental course, yet 

over one third of those recommended to developmental education in a 
given subject did not take any developmental courses in that subject. 

Half of the summer/fall 2004 entering cohort enrolled in at least one developmental 
education course. As is evident from Figure 1, the rate of developmental enrollment was 
particularly high in math, with 43 percent of students taking at least one developmental 
course in that subject. Smaller proportions of students took developmental courses in writing 
(21 percent) or reading (14 percent). A similar pattern of enrollment in developmental 
education (highest for math, followed by writing and then reading) was evident for students 
in transfer and career tech programs, and for students who did and did not earn dual 
enrollment credits during high school. However, transfer program students enrolled in 
developmental math at a higher rate than career-technical students, perhaps due to program 
requirements. In addition, the 9 percent of students who earned dual enrollment credits 
during high school seemed more academically prepared: only 37 percent enrolled in 
developmental courses, compared with 52 percent of students without dual enrollment 
credits. 



Figure 1: Enrollment in Developmental Courses 




However, as shown in Figure 2, only about 50 to 60 percent of students referred to 
developmental education took the same developmental course to which they were 
recommended. While some students took a developmental course other than the one 
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recommended, over a third did not take any developmental course in the recommended 
subject. Thirty-nine percent of students who were recommended to take a developmental 
math course did not do so. Similarly, 35 percent of students who were recommended to take 
a developmental writing course and 41 percent who were recommended to take a 
developmental reading course did not take any developmental courses in those subject areas. 



Figure 2: Enrollment in Developmental Courses 
by Placement Recommendation 
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Finding: Most students did not complete the recommended developmental 

sequence, both because they did not enroll in recommended courses 
and, to a lesser extent, because they did not pass the developmental 
courses that they took. 

Figure 3 shows the rate at which students who placed into transfer programs completed the 
full sequence of developmental courses in a given subject, with each bar representing 
students referred to a particular level of developmental coursework in that subject. Most 
community colleges in Virginia offer three levels of developmental math (labeled “highest,” 
“middle,” and “lowest”), and two levels of developmental reading and writing (labeled 
“highest” and “lowest”). The majority of transfer-placed students did not complete the 
recommended developmental sequence, both because they did not enroll in recommended 
courses and, to a lesser extent, because they did not pass the developmental courses they 
took. Students referred to the lowest levels of developmental instruction fared particularly 
poorly. For example, among students referred to the lowest level of developmental math, 47 
percent did not enroll in any developmental math course; 43 percent enrolled in one or more 
developmental math courses but did not pass one of those courses, thereby never completing 
the sequence; and only 10 percent successfully completed the developmental sequence. In 
contrast, among students referred to the highest level of developmental mathematics (who 
required only one course to complete the sequence), 60 percent did not enroll in the required 
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course; 16 percent enrolled, but did not pass the course; and 24 percent completed the course. 
At all levels and in every subject, transfer-placed students did not complete the 
developmental sequence largely because they did not enroll in the recommended courses, not 
because they did not pass the developmental courses they enrolled in. 



Figure 3: Developmental Enrollment and Performance* 
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Finding: 



Gatekeeper course enrollments (within 4 years) were low, especially in 
math. Gatekeeper course enrollments varied by the level of 
developmental course enrollment, but not by whether or not students 
complied with developmental recommendations. 

Among all students in the summer/fall 2004 entering cohort, less than two thirds (62 percent) 
enrolled in gatekeeper English, and little more than a third (36 percent) enrolled in 
gatekeeper math (see Figure 4). Whether students enrolled in gatekeeper courses depended 
on whether they took developmental courses, and the level of the course they took (see 
Figure 5, which shows the proportion of English gatekeeper enrollment by reading and 
writing recommendation, and the proportion of math gatekeeper enrollment by math 
recommendation). Among developmental students, those who started at lower levels of 
developmental coursework (“Dev - Fowest”) were much less likely to take gatekeeper 
courses than either those who started at the highest level of developmental coursework (“Dev 
- Highest”) or those who did not take any developmental courses in the subject (“No Dev”). 
The outcomes were particularly poor for students who started in the lowest level of 
developmental math (pre- algebra): only 19 percent of them eventually enrolled in gatekeeper 
math. Students who started at the highest level of developmental coursework in reading and 
writing had relatively similar rates of taking gatekeeper English as those who took no 
developmental courses, while students who started in the highest level of developmental 
math course had a higher rate of gatekeeper course enrollment than did students who took no 
developmental coursework. We investigated this issue further and learned that the pattern 
emerged because the “no developmental enrollment” category includes two distinct groups of 
students: those who were recommended to take a developmental math course but did not, and 
those who were not recommended to take developmental math. The first group had a 
relatively low rate of enrollment in gatekeeper math (35 percent) while the second had a 
much higher rate (75 percent). Since the first group is much larger, the overall percentage of 
participation in gatekeeper math is low. 



Figure 4: Enrollment in Gatekeeper Courses 
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Figure 5: Enrollment in Gatekeeper Courses 



c 100% 

<D 



I 80 % - 

n 




Reading Writing Math 



■ No Dev □ Dev - Highest 

■ Dev -Middle □ Dev - Lowest 



Figure 6 compares gatekeeper enrollment rates among students who were referred to college- 
level courses (“College-Level”), those recommended to developmental education who took 
at least one such course (“Dev - Took”), and those recommended to developmental 
education who skipped the developmental sequence (“Dev - Skipped”). Students who were 
not recommended to take a developmental course in a given subject were far more likely to 
enroll in a gatekeeper course for that subject. Yet among students who were recommended to 
take developmental instruction, the rate at which they enrolled in gatekeeper courses did not 
differ much depending on whether or not they complied with the placement recommendation. 
In fact, among students referred to developmental math, a somewhat higher percentage 
enrolled in gatekeeper math without taking the recommended developmental coursework 
than did those who followed the recommendation and took a developmental course (35 
percent versus 31 percent). 
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Figure 6: Enrollment in Gatekeeper Courses 
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Finding: A substantial proportion of students with high placement test scores 

did not take gatekeeper courses. 

Students with higher placement test scores were more likely to take gatekeeper courses than 
were students with lower test scores. However, a substantial number of students with high 
test scores did not take gatekeeper courses. For example, for students scoring in the top 
quartile of reading and writing, between 15 and 20 percent did not take gatekeeper English. 
Among students in the highest quartile of the pre-algebra test, only 3 1 percent took a 
gatekeeper math course, although higher percentages of students in the highest quartiles of 
algebra (65 percent) and college algebra (75 percent) took gatekeeper math. The same pattern 
holds for placement recommendations. Thus, even among students who were arguably well 
prepared for college-level work (based on their test scores and recommendations), many 
were not proceeding to gatekeeper courses. Understanding why students, particularly those 
with high test scores, are not proceeding to gatekeeper courses warrants further investigation. 

Finding: Placement test scores in reading and writing did not predict whether 

students passed gatekeeper English. Math test scores had a stronger 
association with passing gatekeeper math. 

Figure 7 shows the predicted probability of passing gatekeeper English by reading test score 
quartiles for students placed in transfer programs. We estimated these probabilities using 
regressions to control for individual student characteristics and institutions attended. The 
probabilities are virtually identical across test score quartiles, indicating that there is no 
relationship between reading test scores and whether students passed gatekeeper English. A 
similar pattern was found for writing. Math placement test scores seem to predict outcomes 
better than reading and writing scores. The predicted probability of passing gatekeeper math 
was 9 percentage points higher for students in the highest (4 th ) quartile of the pre-algebra test 
score distribution than for those in the lowest (1 st ) quartile. There is a similar gap (8 
percentage points) between students in the highest and lowest algebra test score quartiles. 
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Figure 7: Predicted Probability of Passing Gatekeeper English, by Reading 
Test Score Quartiles, for Students Placed in Transfer Programs 
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Finding: The proportion of students completing gatekeeper courses was low, 

especially for students referred to the lowest level of developmental 
education. However, the low overall completion rate emerged largely 
because students did not enroll in these courses. Among those who did 
enroll in gatekeeper courses, pass rates were fairly high. 

Only 47 percent of students in the summer/fall 2004 entering cohort completed gatekeeper 
English within four years, and a mere 26 percent completed gatekeeper math. Focusing on 
those students who enrolled in developmental education, Figure 8 shows the proportion of 
students in transfer programs who never enrolled in the corresponding gatekeeper course, 
who enrolled but did not pass, and who eventually passed the gatekeeper course (with a grade 
of C or better). Gatekeeper completion rates were especially low among transfer-placed 
students referred to the lowest level of developmental instruction. For example, only 8 
percent of students referred to the lowest-level developmental math course completed 
gatekeeper math, compared with 46 percent of students in the highest level of developmental 
math. 
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Figure 8: Gatekeeper Enrollment and Performance* 
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In general, low overall completion rates emerged largely because students did not enroll in 
gatekeeper courses. Over one third of students did not attempt to take gatekeeper English and 
two thirds did not enroll in gatekeeper math. Non-enrollment in gatekeeper courses was even 
more pronounced among students who were recommended to take developmental courses. 
Students who did enroll in gatekeeper courses had a reasonably high degree of success: 77 
percent of students passed gatekeeper English with a grade of C or higher, and 73 percent 
passed gatekeeper math. Similar patterns held for students placed in transfer and career tech 
programs. This result is consistent with findings from research by CCRC on colleges 
involved with the Achieving the Dream initiative, a national community college reform effort 
(Bailey, Jeong, & Cho, 2009). Thus, for community colleges in Virginia and elsewhere, a key 
challenge is motivating students to enroll in gatekeeper courses in the first place. 
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Finding: 



Gatekeeper course pass rates were similar across students who 
enrolled in different levels of developmental education. Gatekeeper 
course pass rates also did not vary strongly by whether or not students 
complied with their developmental course recommendation. 

Among students who enrolled in a gatekeeper course, rates of passing (with a C or better) did 
not differ markedly between those who previously took developmental courses and those 
who did not (see Figure 9). About three quarters of students who took no developmental 
reading or writing but later enrolled in gatekeeper English were able to pass the course with a 
C or better. Similar percentages of students who enrolled in different levels of developmental 
coursework in reading or writing completed gatekeeper English. Even in math, where gaps in 
gatekeeper enrollment were more pronounced, differences in gatekeeper passing rates 
between students who did and did not take developmental math were relatively small. 
Moreover, gatekeeper passing rates did not differ by whether students followed their 
placement recommendation. Among students recommended to developmental education who 
later enrolled in gatekeeper courses, those who complied with developmental 
recommendations had similar rates of passing with a C or better compared to those who did 
not comply (Figure 10). 



Figure 9: Performance in Gatekeeper Courses 
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Figure 10: Performance in Gatekeeper Courses 
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These findings seem to conflict with one other. The finding that students who took 
developmental courses did about as well in gatekeeper course as those who did not (Figure 9) 
might be interpreted to indicate that developmental education enables those who complete it 
to succeed in gatekeeper courses. Yet, the finding that students who were recommended to 
developmental education but skipped it did as well as those who took developmental courses 
(Figure 10) suggests that developmental instruction does not make a difference. 

In fact, these findings cannot indicate whether or not developmental education is effective. 
CCRC was able to determine that there is a correlation between student behaviors (such as 
taking developmental courses or complying with a developmental recommendation) and 
success in gatekeeper courses; but this does not imply a causal relationship. It may be that 
students who are referred to developmental instruction but nevertheless skip it differ from 
students who enroll in developmental coursework in their level of motivation or other 
unobserved characteristics. Similarly, students who take and complete a developmental 
sequence may be especially motivated or otherwise different in ways that would explain the 
fact that they do as well in gatekeeper courses as students who do not participate in 
developmental instruction. The results do indicate that it would be useful to explore why 
some students decide to forego developmental instruction and are nevertheless successful in 
completing gatekeeper courses and other educational outcomes. 
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Finding: There was substantial variation among Virginia community colleges in 

the rates at which students enrolled in and passed developmental and 
gatekeeper courses. 

Among Virginia’s community colleges, the proportion of entering students who enrolled in 
developmental reading ranged from 5 percent to 25 percent. Similar variation was evident for 
developmental writing and math. The proportion of students enrolling in gatekeeper English 
ranged from 50 percent to 71 percent; the range for gatekeeper math was even wider, from 17 
percent to 50 percent. 

Among students who enrolled in gatekeeper English, pass rates ranged from an average of 66 
percent to 85 percent; average math pass rates ranged from 58 percent to 89 percent across 
institutions. The correlation between gatekeeper enrollment and pass rates at the institutional 
level was very weak, indicating that some institutions are more successful at getting students 
to take these courses, while others are more successful at getting them to pass the courses. 
Similarly, some institutions are better at helping students take and pass gatekeeper English 
while others are more effective at getting students to take and pass gatekeeper math. These 
differences indicate that every Virginia community college has something to offer as well as 
something to learn from the others in the system. 

Finding: Similar patterns were observed with respect to other academic 

outcomes. 

The CCRC study examined an array of other academic outcomes, including attempting and 
accumulating college credits, earning certificates or associate degrees, and transferring to 
four-year institutions. Results for these academic outcomes followed similar patterns as those 
for gatekeeper success: students who enrolled in lower levels of developmental instruction 
did more poorly, and students who did not comply with developmental recommendations did 
just as well as those who complied. For example, only 18 percent of students starting in a 
pre-algebra course (the lowest level of developmental math) earned a certificate or associate 
degree or transferred to a four-year institution, compared with 39 percent of students who 
started math in Algebra II or higher. Similar differences in success rates, though perhaps not 
so stark as in math, were evident in reading and writing. 

Taking and passing college-level English and math are only two of several milestones on the 
way to success in college. Still, they are important both because they are generally required 
for degree programs and because their attainment is associated with increased chances of 
earning a credential or transferring to four-year institutions. Given that similar patterns are 
observed in other key outcomes, it is important to better understand why some students take 
and pass gatekeeper courses while others do not, and to identify strategies colleges can use to 
increase students’ success in these gatekeepers and beyond. 
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Recommendations 



Based on the findings from the study summarized here, CCRC made recommendations to the 
Virginia community colleges toward the goal of improving the success of academically 
unprepared students in gatekeeper courses. The recommendations presented here are those 
that may be applicable to colleges outside Virginia. 

Recommendation: Colleges should investigate why students recommended to take 
developmental instruction do not take these courses. 

CCRC’s analyses of developmental course-taking in Virginia’s community colleges likely 
underestimate the extent of the need for remediation, as some schools were missing large 
amounts of placement recommendation data. Yet even the available data indicate that over 
one third of students recommended for a developmental course in reading, writing, or math 
did not take a single developmental course in that subject. Colleges with similar patterns of 
non-compliance should ask students (through focus groups and surveys) why they choose to 
circumvent these requirements. 

Recommendation: Since students who did not follow their placement 

recommendations fared as well as students who were 
recommended for and took developmental courses, colleges 
should investigate this phenomenon further and learn from 
students about alternative strategies for success. 

Students who were recommended for but did not take developmental courses fared equally 
well (with respect to taking and passing gatekeeper courses and the other educational 
outcomes examined) as did students who were recommended for and enrolled in 
developmental courses. This finding suggests that failing to follow placement 
recommendations is not always detrimental to student success. It is important to note that the 
finding does not imply that developmental instruction is not effective or not needed. Students 
who skip developmental courses may differ from those who enroll in them with respect to a 
number of unobserved characteristics, which may account for their higher success rates. 
However, these results suggest that it would be useful to explore why some students forego 
developmental instruction and are nevertheless successful. 

Recommendation: When considering the effectiveness of developmental instruction 
or developing policies and practices to facilitate student success, 
colleges should consider the level of developmental courses 
taken by students instead of grouping all students in need of 
remediation together. 

Not all Virginia community college students who required remediation fared the same. 
Analyses separating students by level show that students who started community colleges in 
the lowest level of developmental courses did less well with respect to all outcomes 
examined, including completing developmental courses, completing gatekeeper courses, 
accumulating credits, or earning credentials. These patterns persisted even after controlling 



CCRC: VCCS Gatekeeper Summary Report 



13 




for individual characteristics, including test scores and institutions attended. Even when we 
controlled for student demographics and other factors that might bear on success (see the full 
report for details), the gaps between students starting in the lowest level of developmental 
courses and other students were notable, particularly for math. 

Recommendation: Colleges might consider recommending alternative enrollment 
pathways for students in the lowest level of developmental 
courses. 

Given the low rates of success for students recommended to the lowest developmental 
courses on entry, colleges may want to consider alternative approaches to facilitating their 
educational success. For example, colleges may consider encouraging such students to enroll 
in occupational certificate programs that do not require college-level math and English as an 
intermediate step toward eventually earning a degree. Community colleges in Washington 
State have seen promising results from programs that enable adult basic skills students (many 
of whom are at a level of readiness similar to that of the lowest level developmental students) 
to enter and succeed in occupational certificate programs. In the approach developed by the 
Washington colleges, known as I-BEST, adult basic skills students enroll in college-level 
career technical programs that are jointly taught by basic skills and career- technical 
instructors (Jenkins, Zeidenberg, & Kienzl, 2009). 

Recommendation: Colleges should consider surveying students to learn why so 
many are not enrolling in gatekeeper courses and develop 
policies to motivate gatekeeper enrollment. 

Many students examined in this study did not complete gatekeeper courses, not because they 
did not pass these courses, but because they never enrolled in them. Approximately one third 
of students never enrolled in gatekeeper English and two thirds never attempted gatekeeper 
math. Low gatekeeper enrollments were apparent for all groups of students — those with 
different test scores, those in academic transfer and career-technical programs, and those 
taking different levels of developmental courses — but rates were particularly low among 
students starting in the lowest level of developmental courses. Students who did enroll in 
gatekeeper courses had a relatively high degree of success: 77 percent passed gatekeeper 
English and 73 percent passed gatekeeper math. Moreover, developmental students who 
enrolled in gatekeeper courses were approximately equally likely to succeed regardless of 
their starting point. 

It is not clear whether students’ lack of enrollment in gatekeeper courses was due to limited 
capacity to enroll students who needed such courses, problems with scheduling, lack of 
counseling and/or of students’ understanding of the courses they needed to take, or other 
reasons. Surveying students to understand why they are not enrolling in these courses would 
help to illuminate factors deterring students from gatekeeper courses and inform the 
development of policies to motivate and facilitate enrollment in gatekeeper courses. 
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Recommendation: Colleges should explore strategies for increasing enrollment in 
gatekeeper math, given the low rate of participation in such 
courses. 

While not taking gatekeeper math courses was an issue for many students, it was particularly 
pronounced among students in need of remediation (whether that is defined by test scores, 
placement recommendations, or actual enrollment in developmental courses). Given low 
gatekeeper math enrollment rates, colleges should explore strategies for increasing 
enrollment in math, which may entail working with high schools to improve math 
preparation, providing summer bridge programs, altering the structure of developmental 
instruction in math, or providing alternatives to College Algebra that teach students 
mathematics relevant to the degree programs and career fields they are seeking to enter. 

Recommendation: Colleges might investigate whether ‘'mainstreaming” some 
students, particularly those referred to the highest level of 
developmental coursework in a given subject area, directly into 
college-level courses, while providing additional supports as 
needed, is an effective strategy for facilitating their educational 
attainment. 

Some students referred to developmental education chose to circumvent those 
recommendations, and yet they were able to succeed at a rate similar to those that complied 
with placement recommendations. Placement tests do not purport to capture the complete 
range of factors that allow students to succeed (for example, personal motivation and 
interpersonal supports). Accordingly, it may be appropriate to allow some students who score 
below the college-ready threshold to attempt college-level classes, particularly if such 
students are supplied with systematic academic and non-academic supports. Programs that 
attempt to accelerate the progress of remedial students into college-level courses by offering 
developmental instruction concurrently with related college-level courses or by integrating 
academic support into college courses have shown some promise (see, e.g., Bragg & Barnett, 
2009; Scott, 2003; Wlodkowski 2003; Wlodkowski & Kasworm, 2003), although these 
approaches have not yet been rigorously evaluated (Bailey, 2009). 

Recommendation: Colleges should review policies and practices aimed at 

increasing gatekeeper course completion among college-ready 
students, and look for promising practices both within and 
outside the system. 

Even among students who were reasonably well prepared academically, many did not make 
good progress toward a degree. Over 15 percent of students in the top quartiles of the reading 
and writing tests did not enroll in gatekeeper English, and 25 percent of students in the top 
quartile of the college algebra test did not enroll in gatekeeper math. This finding indicates 
that academic preparation (or lack thereof) is not the only factor that needs to be addressed to 
improve gatekeeper enrollment rates. Understanding why even students who are considered 
prepared for college-level work are not achieving key milestones is an important step in 
enhancing overall success rates. It is likely that policies designed to assist academically 
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prepared students to complete gatekeeper courses will also be beneficial for students in need 
of remediation. 

Recommendation: Colleges should examine the practices of their peers that are 
more effective in enabling student success and should share 
promising practices. State systems should facilitate this 
exploration and sharing. 

There was much variation across the Virginia community colleges in the rates at which 
students enrolled and passed gatekeeper courses. For example, the average proportion of the 
cohort who enrolled in gatekeeper math in the four-year study period ranged from 17 percent 
in one school to 50 percent in another. In addition, some institutions are more successful at 
getting students to enroll in gatekeeper courses, while others are more successful at getting 
enrolled students to pass them. Similarly, some institutions are better at helping students 
progress through gatekeeper English, while others are more effective in terms of gatekeeper 
math. Thus all institutions have something to offer as well as something to learn from the 
others. 

To provide research-based guidance on this issue, CCRC is partnering with VCCS on a 
study, funded by the Bill and Melinda Gates Foundation, of colleges identified through 
quantitative analysis as comparatively effective (controlling for student characteristics and 
other factors) in enabling students to take and pass gatekeeper English and math. VCCS has 
also received funding from the Gates and Lumina foundations through the Achieving the 
Dream initiative to organize discussion of college-level data on student attainment of key 
milestones and sharing of promising practices for improving student success among colleges. 
Five other states (Connecticut, Florida, North Carolina, Ohio, and Texas) are involved in the 
initiative and are engaged in similar activities. The approach these states are following has 
the potential to promote systemic changes that benefit large numbers of students and to 
inform changes in state policy that provide colleges with resources and incentives to continue 
to innovate over time. For this reason, other states should consider following their lead. 
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